FARMER: A novel approach to file access correlation mining and evaluation reference model     

夏鹏  冯丹  王芳 《上海大学学报(英文版)》2011,15(6):574-588

File semantic has proven effective in optimizing large scale distributed file system. As a consequence of the elaborate and rich I/O interfaces between upper layer applications and file systems, file system can provide useful and insightful information about semantic. Hence, file semantic mining has become an increasingly important practice in both engineering and research community. Unfortunately, it is a challenge to exploit file semantic knowledge because a variety of factors could affect this information exploration process. Even worse, the challenges are exacerbated due to the intricate interdependency between these factors, and make it difficult to fully exploit the potentially important correlation among various semantic knowledges. This article proposes a file access correlation miming and evaluation reference (FARMER) model, where file is treated as a multivariate vector space, and each item within the vector corresponds a separate factor of the given file. The selection of factor depends on the application, examples of factors are file path, creator and executing program. If one particular factor occurs in both files, its value is non-zero. It is clear that the extent of inter-file relationships can be measured based on the likeness of their factor values in the semantic vectors. Benefit from this model, FARMER represents files as structured vectors of identifiers, and basic vector operations can be leveraged to quantify file correlation between two file vectors. FARMER model leverages linear regression model to estimate the strength of the relationship between file correlation and a set of influencing factors so that the “bad knowledge” can be filtered out. To demonstrate the ability of new FARMER model, FARMER is incorporated into a real large-scale object-based storage system as a case study to dynamically infer file correlations. In addition FARMER-enabled optimize service for metadata prefetching algorithm and object data layout algorithm is implemented. Experimental results show that is FARMER-enabled prefetching algorithm is shown to reduce the metadata operations latency by approximately 30%–40% when compared to a state-of-the-art metadata prefetching algorithm and a commonly used replacement policy.  相似文献   

基于工作流的分布式数据仓库模型的研究     

李玥  杨成全 《雁北师范学院学报》2011,27(3)

通过对数据仓库和分布式数据仓库体系构架的分析,得出当前分布式数据仓库存在的缺点：不能进行灵活的定制,易用性差。分布式数据仓库为了保持全局数据仓库和局部数据仓库的数据一致性,需购买大量昂贵的存储设备来保存两份相同的数据。针对这些缺点,提出了一种基于工作流的分布式数据仓库模型,模型的整体结构基于SOA面向服务的体系结构、分布式。引入了工作流技术来实现灵活定制数据提取需求、分析挖掘需求及结果可视化的需求,并将这些功能封装成整体模型中的业务流程服务。在业务流程服务模型部分提出了数据仓库业务流程执行语言、业务流程的可达性验证算法和优化算法。  相似文献   

Web日志挖掘技术在校园网信息处理中的应用研究     

张琳 《遵义师范学院学报》2011,13(5):112-115

随着校园网的快速发展,产生了海量信息,怎样利用数据挖掘技术根据校园网用户的访问日志挖掘出有价值的信息,是值得深入研究的。作者首先介绍了Web日志挖掘技术的主要内容,再深入研究在校园网信息处理方面的应用,重点以遵义师范学院校园网为实例,使用关联规则进行研究分析,探讨出用户的兴趣与偏好,从而更好地为学校教学管理服务。  相似文献   

C语言中变量的性质     

田艳华 《中国科技信息》2006,(5):60-61

讨论了C语言中变量的特性，分析了变量的分类、作用域、存储类型厦存储方式。  相似文献   

Automatic ranking of information retrieval systems using data fusion     

Rabia Nuray  Fazli Can 《Information processing & management》2006

Measuring effectiveness of information retrieval (IR) systems is essential for research and development and for monitoring search quality in dynamic environments. In this study, we employ new methods for automatic ranking of retrieval systems. In these methods, we merge the retrieval results of multiple systems using various data fusion algorithms, use the top-ranked documents in the merged result as the “(pseudo) relevant documents,” and employ these documents to evaluate and rank the systems. Experiments using Text REtrieval Conference (TREC) data provide statistically significant strong correlations with human-based assessments of the same systems. We hypothesize that the selection of systems that would return documents different from the majority could eliminate the ordinary systems from data fusion and provide better discrimination among the documents and systems. This could improve the effectiveness of automatic ranking. Based on this intuition, we introduce a new method for the selection of systems to be used for data fusion. For this purpose, we use the bias concept that measures the deviation of a system from the norm or majority and employ the systems with higher bias in the data fusion process. This approach provides even higher correlations with the human-based results. We demonstrate that our approach outperforms the previously proposed automatic ranking methods.  相似文献   

基于COM的关联规则挖掘算法的设计与实现     

尹燕 《科技广场》2007,(5):87-88

本文首先从理论角度分析了关联规则挖掘算法与聚类挖掘算法原理及其应用领域,然后介绍了Aprior算法的实现及封装,并设计了可视化界面,对算法进行了测试。  相似文献   

文本挖掘与中文文本挖掘模型研究   总被引：5，自引：0，他引：5  

谌志群  张国煊 《情报科学》2007,25(7):1046-1051

文本挖掘，又称为文本数据挖掘或文本知识发现，是指在大规模的文本中发现隐含的、以前未知的、潜在有用的模式的过程。本文首先对文本挖掘进行了概述，给出了文本挖掘的定义、特点和研究现状。然后对国内中文文本挖掘的研究现状进行了分析，指出了当前中文文本挖掘研究中存在的主要问题和主要研究方向。最后提出了一个统一的中文文本挖掘模型——UCTMF。该模型具有层次性、开放性和可扩展性，为中文文本挖掘系统提供了基本体系框架。  相似文献   

数据加密中的密钥管理和协商技术     

龚力 《中国科技信息》2006,6(24):113-113,116

在介绍了XML的岳全特性和其加密规范后，从Dfffie-Heliman密钥协商的生成原理出发，运用XML加密规范所提供的对数据对象进行密钥协商的机制，并以经典的D—H密钥协商为例，依据规范所制定的原则，运用JCE所具有的安全特性为实现技术，就如何时密钥协商交换进行了探讨，从而为XML加密规范的实际应用探索了一种方式，并以此说明建立自主加密扩展的重要意义。  相似文献   

基于数据挖掘的网络信息采集与服务研究     

陈征华  杨内 《情报理论与实践》2007,30(5):702-704

本文采用数据挖掘技术,构建一种网络信息采集与服务模型,从互联网上将电子行业相关信息挖掘出来,生成一系列独具军工电子特色的专业/专题数据库,并建设电子科技文献网络采集数据库,满足各相关用户对网络信息资源的需求,同时解决了困扰用户的网络海量信息利用和异构信息索取问题,减轻了用户鉴别信息可靠性的时间,提高了网络信息资源的利用率。  相似文献   

大数据时代高校学报数字化转型研究——以贵州省为例     

谢鸣 《中国科技期刊研究》2016,27(9):957-962

【目的】以贵州省为例,明晰贵州省高校学报数字化发展的现状以及存在的主要问题,提出建设建议。【方法】主要通过网络搜索和实地走访,对贵州省高校学报的数字化发展进行调研和分析。【结果】利用大数据技术,以省域为单位建立贵州省学报数据共享管理平台,并以“互联网+高校学报大数据”的创新模式,同步构建贵州省高校学报共享管理平台APP,通过两者的深度融合,实现共赢发展,扩大贵州省高校学报的影响力。【结论】大数据运用于贵州省高校学报数据资源,可以促进学报的数字化转型。  相似文献